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Abstract. The random A-satisfiability (A"-SAT) problem is an important problem for 
studying typical-case complexity of NP-complete combinatorial satisfaction; it is also a 
representative model of finite-connectivity spin-glasses. In this paper we review our recent efforts 
on the solution space fine structures of the random A-SAT problem. A heterogeneity transition 
is predicted to occur in the solution space as the constraint density a reaches a critical value a cm - 
This transition marks the emergency of exponentially many solution communities in the solution 
space. After the heterogeneity transition the solution space is still ergodic until a reaches a larger 
threshold value ad, at which the solution communities disconnect from each other to become 
different solution clusters (ergodicity-breaking). The existence of solution communities in the 
solution space is confirmed by numerical simulations of solution space random walking, and 
the effect of solution space heterogeneity on a stochastic local search algorithm SEQSAT, which 
performs a random walk of single-spin flips, is investigated. The relevance of this work to glassy 
dynamics studies is briefly mentioned. 



1. Introduction 

A random iT-Satisfiability (K-SAT) formula is constructed by adding M constraints on N 
variables pQ. Each variable i has a spin state at = ±1, and each constraint a applies to K 
different variables that are randomly chosen from the whole set of variables. The energy of 
constraint a is expressed as 



where da denotes the set of variables involved in constraint a (the size of da is K), J* = ±1 is 
the preferred spin state of constraint a on variable i. Each of the K preferred spin values J* of a 
constraint a are randomly and independently assigned a value +1 or —1 with equal probability. 
The whole set { J*} of preferred spin values are then fixed, but the actual spin state Oi of each 
variable i is allowed to change. The constraint energy E a is zero if at least one of the variables 
i E da takes the spin value a% = J*, otherwise E a = 1. Given a random JC-SAT formula, the 
task is to construct at least one spin configuration a = {a\, <T2, . . . , ctjv} that satisfies all the 
constraint (i.e., makes all the constraint energy E a to be zero), or to prove that no such solutions 
(i.e., satisfying spin configurations) exist. 
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When N is large, rigorous mathematical proofs (see, e.g., review article [2]) and numerical 
simulations [JJ revealed that whether a random if-SAT formula is satisfiable or not depends 
on the constraint density a (= M/N). As a increases beyond certain satisfiability threshold 
a s (K), the probability of a randomly constructed if -SAT formula to be satisfiable quickly drops 
from being close to unity to being close to zero. Among the whole ensemble of random if-SAT 
formulas, the satisfiability of those instances with constraint density a in the vicinity of a s {K) 
is the hardest to determine. This empirical observation has stimulated a lot of investigations. 

The random K-SAT problem was intensively studied in the statistical physics community 
during the last decade [3l HI El [7] . The number of solutions for the random K-SAT problem 
as a function of constraint density was calculated in Ref. [3] by the replica method of spin-glass 
physics. Later the satisfiability threshold a s as a function of if was calculated by the first-step 
replica-symmetry-breaking (1RSB) energetic cavity method [HIS]. The evolution of the solution 
space of the random if-SAT problem with constraint density a was studied in Refs. [51 [10] 
using the 1RSB entropic cavity method [HI Before the satisfiability threshold a s (K) is 
reached, the solution space experiences a clustering transition at a = ctd(K), where exponentially 
many solution clusters (Gibbs states) form and the ergodicity of the solution space is broken. 
This clustering transition is followed by a condensation transition at a = a c (K) > atd(K), where 
a sub-exponential number of solution clusters begin to dominate the solution space. Within a 
solution cluster, the spin states of a large fraction of variables start to be frozen to the same 
value as a exceeds certain threshold value a/ that may be different for different clusters. The 
freezing transition was investigated in Refs. [9j[T3j[Tl]. Some of these phase transitions (i.e., the 
clustering and the condensation transition) were earlier found to occur in mean-field p-body- 
interaction spin glasses |15j . with temperature T (instead of the constraint density) being the 
control parameter. 

In the present paper, we review our recent efforts on the solution space fine structures of 
the random if-SAT problem. A heterogeneity transition is predicted to occur in the solution 
space as the constraint density a reaches a critical value a cm (if ) which is smaller than ad(K) 
|16| . This transition marks the emergency of exponentially many solution communities in the 
solution space. For a cm < a < ad, the heterogeneous solution space is ergodic; at a = ad the 
solution communities will turn into different solution clusters as an ergodicity-breaking transition 
occurs. The existence of solution communities in the solution space is confirmed by numerical 
simulations on single if -SAT formulas [T7j, and the effect of solution space heterogeneity on a 
stochastic local search algorithm SEQSAT, which performs a random walk of single-spin flips, is 
investigated [18]. Beyond the clustering transition point a^, our numerical simulation results 
[T71 [T9] suggested that the individual solution clusters of the solution space also have rich internal 
structures. 

The replica-symmetric cavity method is used in the next section to calculate the value of a cm 
for the onset of solution space structural heterogeneity. Section [3] presents the data-clustering 
results on sampled solutions of single random if -SAT formulas; Section [4] reports the simulation 
results of the stochastic search algorithm SEQSAT. We conclude this work in Sec. [5] and point out 
some possible links with the phenomena of two-step relaxation and dynamical heterogeneity in 
supercooled liquids. 

2. Solution space heterogeneity transition 

Given a random if-SAT formula, the total energy E(a) = J2 a E a is equal to the number of 
violated constraints by the spin configuration a. The whole set S of spin configurations with 
E(a) = form the solution space of this if -SAT formula. The similarity between any two 
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Figure 1. Detecting heterogeneity of the solution space by the shape of the entropy density 
function s(q): At low constraint density a (left panel), s(q) is a concave function of the solution- 
pair overlap q. Solution communities start to form as a exceeds a threshold value a cm (middle 
panel) and the solution space becomes heterogeneous; this makes s(q) to be non-concave. At the 
clustering transition point a<2 (right panel), solution communities separate into different clusters, 
and s(q) becomes non-monotonic. As a further increases, solution-pairs with intermediate 
overlap values may cease to exist and then s(q) = — oo for these intermediate q values. 



solutions ct 1 and a 2 of the space S can be measured by an overlap parameter defined as 

9(*V) = ^X>W- ( 2 ) 



To characterize the statistical property of the solution space, we count the total number of 
solution-pairs with overlap value q and denote this number as M(q). For a random if-SAT 
formula of size N > 1 and constraint density a < a s (K), the size of S is exponential in 
N, and J\f(q) is also exponential in N. It is helpful to define an entropy density s(q) as 
s(q)^(l/N)ln[M(q)]. 

The solution space S has a natural graphical representation (solution graph), in which each 
solution is denoted as a node and two nodes are connected by an edge if (and only if) the 
corresponding solutions are interchangeable by a single-spin flip. When a is less than the 
clustering transition point a^{K), this solution graph has only a single connected component 
(solution cluster) that contains all (or almost all) of the solutions [5] . As is well known in social 
and biological network studies (see, e.g., Refs. [201 [21]), a connected network component may 
have rich internal community structures. A community of a graph refers to a subset of nodes of 
the graph that are much more densely inter-connected with each other than with the remaining 
nodes of the graph. If communities exist in the solution graph of a large random iT-SAT formula, 
many subsets of solutions (solution communities) can then be extracted, with the property that 
each solution community contains a set of solutions that are much more similar with each other 
than with the other solutions. If such solution communities are abundant and of considerable 
sizes, they will be statistically relevant and will be manifested in the form of the entropy density 
function s(q) (see Fig. 

We define a partition function Z{x) as 

Z{x)= E E ^v(Nxq{a\a 2 ))=Y J ^v[N{s(q)+xq)] , (3) 
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Figure 2. The mean overlap q(x) and the susceptibility x( x ) a ^ constraint density value a for 
the random 3-SAT problem. In (A) a increases from 3.72 to 3.76 (right to left) with step size 
0.01. The insets of (B) demonstrate that the peak value of x diverges inverse linearly with a 
and x as the critical point (a cm = 3.7497, x cm = 0.0024) is approached. 

where x is an auxiliary binding field. The free entropy <3?(x) of the system is defined as 
$(x) = lnZ(x). In the thermodynamic limit of N — >■ oo, the free entropy density 4>{x) = $>{x)/N 
is related to the entropy density s(q) by 

4>(x) = max [s(q) + xq] = s(q(x)) + xq(x) . (4) 
?e[-i,l] 

In Eq. Q, q(x) is the mean value of solution-solution overlaps under the binding field x, it is 
the argument value at which the function s(q) + xq reaches the global maximum. 

If the entropy density s(q) is a concave function of q € [qo, 1] (Fig. [TJ left panel), where qo is 
the most probable solution-pair overlap value, then for each x > there is only one mean overlap 
q, and q(x) is a continuous function of x. On the other hand, if s(q) is non-concave in q S [qo, 1] 
(Fig. [T] middle and right panel), then the value of q(x) changes discontinuously at certain value 
of x = x* > 0. We have exploited this correspondence between the non-concavity of s(q) and the 
discontinuity of q(x) to determine the threshold constraint density a cm at which s(q) starts to 
be non-concave [16] . We regard a = a cm as the point at which the solution space of the random 
-RT-SAT problem transits into structural heterogeneity. This is because, as schematically shown 
in Fig. [TJ at a cm it starts to make sense to distinguish between intra-community overlap values 
and inter-community overlap values. As shown in Sec. [3j many solution communities can be 
identified in a heterogeneous solution space [17J. Each solution community contains a set of 
solutions which are more similar with each other than with the solutions of other communities. 
These differences of intra- and inter-community overlap values and the relative sparseness of 
solutions at the boundaries between solution communities cause the non-concavity of s(q). 

For the random if-SAT problem, we use the replica-symmetric cavity method of statistical 
mechanics [UJ to calculate the mean overlap values q(x) at each value of a < otd(K). As the 
partition function Eq. ^ is a summation over pairs of solutions (a 1 , a 2 ), the state of each vertex 
is a pair of spins (a, a'). Details of this calculation can be found in Ref. |16j and here we cite 
the main results. 

Figure [2j^ shows, for the random 3-SAT problem, the form of the function q(x) at several 
different a values. When a < a cm (3) w 3.75, the mean overlap q increases with the binding field 
x smoothly, indicating that the entropy density function s(q) is concave in shape (Fig. [TJ left 
panel). The solution space S of the random 3-SAT problem is then regarded as homogeneous. 



When a > a cm (3), there is a hysteresis loop in the q(x) curve as the binding field x increases and 
then decreases around certain threshold value x* . This behavior is typical of a first-order phase- 
transition. At x > x* the partition function Z(x) is contributed mainly by intra- community 
solution pairs, while at x < x* it is contributed mainly by inter-community solution pairs. 

To determine precisely the critical value a cm , we investigate the overlap susceptibility 
X = dq(x)/dx, which is a measures of the overlap fluctuations, 
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where (...) means averaging over solution-pairs under the binding field x. From the divergence 
of the peak value of xi x ) as shown in Fig. [2^3, we obtain that a cm (3) = 3.7497 for the random 
3-SAT problem. This value is much below the value of 0^(3) = 3.87 [SJ. 

For the random 4-SAT problem, we find that a cm (4) = 8.4746. This value is again much 
below the clustering transition point a^(4) = 9.38 [5]. The difference a^(K) — a cm (K) appears 
to be an increasing function of K. 

3. Solution graph random walking 

For single random K-SAT formulas, the solution space structural heterogeneity can also be 
detected by performing a long-time random walking in the corresponding solution graphs 
[22| 123"! [T9l ITT] . The Hamming distance d(a ,d ) between two solutions a 1 and a 2 of the 
solution space is defined as 

N 

d(a\a 2 )=J2[l-S(a},al)] (6) 

i=\ 

where 5 (a, a') = 1 if a = a' and 5 (a, a') = if a = —a'. This distance counts the number 
of different spins between the two solutions. The Hamming distance d{a l ,a 2 ) is related to the 
overlap q^ 1 ,^ 2 ) [Eq. ^] through 

(7) 

In the solution graph of a satisfiable random iC-SAT formula, a solution a is linked to k$ other 
solutions, all of which have unit Hamming distance with a. It was empirically found that the 
degrees kg of the solutions are narrowly distributed with a mean much less than N [TTJ. The 
solutions can therefore be regarded as equally important in terms of connectivity. However, the 
connection pattern of the solution graph can be highly heterogeneous. Even when the whole 
graph is connected, solutions may still form different communities such that the edge density of 
a community is much larger than that of the whole graph. The communities may even further 
organize into super-communities. 

Consider two solutions a 1 and a 2 of the solution graph. The shortest-path length I (a 1 , a 2 ) 
between these two solutions in the solution graph satisfies the inequality /(cr 1 ,*? 2 ) > d{a l ,a 2 ). 
If a 1 and a 2 belong to the same solution community, l{a l ,a 2 ) may be equal to or just slightly 
greater than the Hamming distance d(a l ,(T 2 ). On the other hand, if a 1 and a 2 belong to two 
different solution communities, an extensive spin rearrangement may have to be made to change 
from a 1 to a 2 by single-spin flips, and then ^(cx 1 , a 2 ) is much greater than d(a l , a 2 ). If solutions 
are sampled by a random walking process at equal time interval At, the sampled solutions 
should contain useful information about the community structure of the solution space, with a 
resolution level depending on At. This is because that, when the edges are followed randomly 
by a random walker, the walker will be trapped in different communities most of the time, and 
the sampled solutions will then form different similarity groups. 



Starting from an initial solution a , a sequences of solutions (er°, a 1 , . . . , a 1 , a +1 , . . .) are 
generated by solution graph random walking. Two different random walking processes are used 
in the simulation. In the unbiased random walking process, the solution a l+1 is a nearest neighbor 
of solution a for I = 0, 1, . . . [T7]. Under this simple dynamics, if the generated solution sequence 
is infinitely long, each solution a in a connected component of the solution graph will appear 
in the sequence with frequency proportional to its connectivity kg. On the other hand, in the 
uniform random walking process, with probability kgi/N, the solution a l+1 (I = 0, 1, . . .) is a 
nearest neighbor of solution a 1 , and with the remaining probability 1 — kgi/N, solution a l+1 is 
identical to a . Under this later dynamical rule, each solution in a connected component of the 
solution graph will have the same frequency to appear in the generated solution sequence. Both 
these two random walking processes were used to sample solutions, and the clustering analysis 
performed on these two sets of data gave qualitatively the same results [17J. The simulation 
results shown in Fig. [3] were obtained by the unbiased random walking process. 

A set of solutions are sampled (with equal time interval At) from the generated long sequence 
of solutions for clustering analysis. (To avoid strong dependence on the input solution a , the 
initial part of the solution sequence was not used for sampling.) By calculating the overlap 
values between the sampled solutions, we obtain an overlap histogram as shown in Fig. [3j A 
hierarchical minimum- variance clustering analysis [23] (see also Ref. |25j ) is performed on the 
sampled solutions. Initially each solution is regarded as a group, and the distance between two 
groups is the Hamming distance. At each step of the clustering, two groups C a and C b that 
have the smallest distance are merged into a single group C c . The distance between the merged 
group C c and another group C d is calculated by 

A(r r , (\C a \ + \C d \)d(C a ,C d ) + (\C b \ + \C d \)d(C b ,C d ) - \C d \d(C a ,C b ) 

d{C - Cd) = \cJ+\Cd\ ' (8) 

where \C\ is the number of solutions in group C. After the sampled solutions are listed in 
the order as reported by the minimal-variance clustering algorithm, the matrix of Hamming 
distances between these solutions are represented in a graphical form as shown in Fig. [3] (upper 
row). 

For a random 4-SAT formula with N = 20, 000 variables, the overlap histogram and the 
Hamming distance matrix of 1000 sampled solutions are shown in Fig. [3] for two different 
constraint density values a = 9.10 and a = 9.22 in the ergodic phase. At a = 9.10, the 
Hamming distance matrix has a weak signature of the existence of many solution communities, 
and the overlap histogram is slightly non-concave. At a = 9.22, the Hamming distance matrix 
has a very clear block structure and the overlap histogram is non-monotonic. These observations 
suggest that the explored solution spaces are heterogeneous both at a = 9.10 and a = 9.22; the 
entropic trapping effect of the solution communities becomes much stronger as a increases from 
9.10 to 9.22. The simulation results are consistent with the analytical results of the preceding 
section, they are also in agreement with the expectation that, as a d {4) is approached from below, 
a random walker becomes more and more trapped in a single solution community and finally 
becomes impossible to escape from this single community at a > a d (4). 

We have obtained similar simulation results on single random 3-SAT formulas [T7J. The 
solution space random walking simulations confirm that the solution space of the random K- 
SAT problem is already very heterogeneous before the clustering transition point a d (K) is 
reached. The simulation results reported in Ref. [17] also suggested that, at a > a d (K), the 
single solution clusters of the solution space of the random K-SAT problem are themselves quite 
heterogeneous in internal structure, which may be the reason underlying the non-convergence 
of the belief-propagation iteration process within a single solution cluster [19J. 
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Figure 3. The matrix of Hamming distances of 1000 sampled solutions for a random 4- 
SAT formula of N = 20, 000 variables, and the corresponding overlap distribution of these 
sampled solutions. The solutions were obtained by an unbiased random walking process (solution 
sampling began after running the random walking process for 3 x 10 7 x N steps starting from 
an initial solution, and two consecutively sampled solutions are separated by At = 50, 000 x N 
random walking steps). The left panel corresponds to a = 9.10 and the right panel to a = 9.22. 
In the Hamming distance matrices, the solutions are ordered according to the results of the 
minimal-variance clustering analysis. 

4. Glassy behavior of a stochastic search algorithm 

After a slight change, the solution graph random walking process of the preceding section was 
turned into a stochastic local search algorithm [18j. This algorithm, referred to as SEQSAT, 
satisfies sequentially the constraints of a random AT-SAT formula F. We denote by F m the 
sub-formula containing the first m constraints of F. Suppose a configuration a(t m ) that satisfies 
F m is reached at time t m . The (m + l)-th constraint of formula F is added to F m to obtain 
the enlarged sub-formula F m+ \. Then, starting from a(t m ), an unbiased random walk process 
is running on the solution graph of F m until a spin configuration er(t TO+1 ) that satisfies F m+ % is 
first reached at time t m +i = t m + At m after At m x N single-spin flips, N being the total number 
of variables in formula F. The waiting time of satisfying the (m + l)-th constraint is At m (this 
time is zero if a(t m ) already satisfies F m+ i). Starting from a completely random initial spin 
configuration c?(0) and an empty sub- formula Fq, every constraint of the formula F are satisfied 




Figure 4. The total search time t{ol) needed to satisfy the first ax N constraints of a random 
K-SAT formula with N = 100,000 variables (K = 3, left panel; K = 4, right panel). The 
performances of an unbiased random-walk search process and a biased random-walk search 
process are compared. The red dashed lines correspond to the jamming transition points 
aj(K) as predicted by a long-range-frustration mean-field theory [261 HE], the green dotted lines 
correspond to the clustering transition points otd(K) [5], and the black solid lines correspond to 
the satisfiability threshold value a s {K) [8]. 

by SEQSAT in this way. Notice that if a constraint was satisfied by SEQSAT, it remains to be 
satisfied as new constraints are added (energy barrier crossing is not allowed). 

Figure [4] shows the simulation results of SEQSAT on a random 3-SAT formula and a random 
4-SAT formula, N = 10 5 . When the constraint density a of the satisfied sub- formula F m is 
low, the waiting time needed to satisfy a constraint is very close to zero. As the heterogeneity 
transition point a cm {K) is reached, however, the waiting time increases quickly and it starts to 
take more than 100 x N single-spin flips to satisfy a constraint. The search process becomes 
more and more slow as a further increases. As a approaches another threshold value ctj(K) 
the waiting time is so long (exceeding 10 6 x N single-spin flips) that SEQSAT essentially stops to 
satisfy a newly added constraint. The parameter ctj(K) is regarded as the jamming point of the 
random walk search algorithm. In the range of a cm (K) < a < ay (if) the SEQSAT is performing 
an increasingly viscous diffusion in the solution space of the random if-SAT formula. The 
simulation results of Fig. [4] and those reported in Ref. [18] for single random 5-SAT and 6-SAT 
formulas clearly demonstrate that, the solution space heterogeneity transition at a cm (K) has 
significant dynamical consequences for stochastic local search processes. 

For the random 3-SAT problem, the jamming point ay(3) is significantly larger than the 
clustering transition point «d(3). At a^(3) the solution space of a large random 3-SAT formula 
is dominated by a sub-exponential number of solution clusters [5] . A dominating solution cluster 
also has internal community structures [IT]- As a further increases, a dominating cluster shrinks 
in size, and it break into many sub-clusters. Figure [4] (left panel) indicates that, the random 
walker of SEQSAT is residing on one of the dominating clusters at a ~ ctd(3) and it continue to 
be residing on one of the dominating sub-clusters of the visited cluster as a increases. If the spin 
values of a large fraction of variables become frozen in the residing solution cluster, a jamming 
transition then occurs. The value of ay (3) is predicted to be 4.1897 by a long-range frustration 
mean-field theory [26\ I18j. in agreement with the simulation results. 

For the random K-SAT problem with K > 4, the simulation results shown in Fig. [4] (right 
panel) and in Ref. |18| suggest that the jamming transition point ctj(K) is identical to or very 
close to the clustering transition point a^(K). We notice that for the random K-SAT problem 



with K > 4, at a = ad(K), the union of an exponential number of small solution clusters is 
contributing predominantly to the solution space [5]. It is expected that, as a approaches otd{K) 
from below, the ergodic solution space of a large A-SAT formula is dominated by an exponential 
number of small solution communities. The solutions reached by SEQSAT probably are residing 
on one of these small communities. Asa-?- atd{K), each of these small (but statistically relevant) 
solution communities probably contains a large fraction of variables that are almost frozen. 

Figure [4] also shows the performances of a biased random walk search algorithm. The biased 
random walk process differs from the unbiased random walk process in that, in each single- 
spin flip, a variable that is flippable but not yet being flipped is flipped with priority [18]. 
This biased random walk SEQSAT algorithm was implemented with the hope of escaping from 
a solution community more quickly. For the random 3-SAT and 4-SAT problem, the biased 
SEQSAT algorithm indeed is more efficient than the unbiased algorithm. But for the random 
A-SAT problem with K > 5, the biased algorithm diverges earlier than the unbiased algorithm 

m 

5. Conclusion and discussions 

The solution space structure of the random A-SAT problem evolves with the constraint density 
q. Several qualitative transitions occurs in the solution space as a becomes relatively large. 
We demonstrated, both theoretically and by computer simulations, that the first qualitative 
structural transition is the heterogeneity transition at a = a cm (K), where exponentially many 
solution communities start to form in the (still ergodic) solution space. The dynamic behavior 
of a stochastic search algorithm was investigated. This simple algorithm SEQSAT constructs 
satisfying spin configurations for a single K-SAT formula by performing a random walk of 
single-spin flips. Due to the entropic trapping effect of solution communities, the solution space 
random walking process starts to be very viscous as a goes beyond a cm (K). SEQSAT is able to 
find solutions for a random A-SAT formula with constraint density less than a threshold value 
ctj(K). For A = 3, the jamming point ay (3) ~ 4.19 is larger than the solution space clustering 
transition point a^(3). But for A > 4, it appears that ctj(K) is very close to a^(A). 

When the constraint density a of a large random A-SAT formula is in the region of 
a cm (A) < a < ad(K), the solution space of the formula is heterogeneous but ergodic. If 
a random walking process is running on the solution space starting from an initial solution, 
a two-step relaxation behavior can be observed |17| . corresponding to a quick slipping into a 
solution community, a relatively long wandering within this community, and finally the viscous 
diffusing in the whole solution space. Similar two-step relaxation behaviors were observed in 
glassy dynamics studies of supercooled liquids (see, e.g., the review article [27]) both in the 
non-activated dynamics region and in the activated dynamics region. 

The existence of solution communities in the solution space may lead to a phenomenon that is 
similar to the dynamical heterogeneity of supercooled liquids (see, e.g., review articles [28, 29]). 
For a random walking process on a heterogeneous solution space, if the observation time is less 
than the typical relaxation time of escaping from the solution communities, one may find that 
the spin values of a large fraction of variables change only very infrequently among the visited 
solutions, while the spin values of the remaining variables are flipped much more frequently. The 
variables of the A-SAT formula then divides into an active group and an inactive group in this 
observation time window. One may further observe that the active variables are clustered into 
many distantly separated sub-groups, each of which containing variables that are very close to 
each other [3D]. Numerical simulations need to be performed to investigate in more detail the 
solution space random walking processes. 

The constraint density a of the random A-SAT problem corresponds to the particle density 
p in supercooled liquid. In supercooled liquids, another important control parameter is the 
temperature T. We can also introduce a positive temperature T to the random A-SAT problem 



and investigate how the configuration space evolves with T. It is anticipated that, as T is lowered 
to a threshold value T cm (K,a) the configuration space will also experience a heterogeneity 
transition |16| . Detailed theoretical and simulation results will be reported in a later work. For 
the fully-connected p-spin spherical model of glasses, it has already been shown |32| [33] that a 
quantity called the Franz-Parisi potential has a change in its concavity property as T is lowered 
(see [33] for more recent extended studies). Another related problem, the weight space property 
of the Ising perceptron, was studied in [35] • For this fully connected model, a change of concavity 
property was also predicted for its characteristic function. 

The solution space heterogeneity transition also exists in the random i^-XORSAT problem 
|18| . It may be a general feature of the configuration spaces of spin glass models on finite- 
connectivity random graphs. 



Acknowledgments 

The author thanks Kang Li, Hui Ma and Ying Zeng for collaborations. This work was partially 
supported by the National Science Foundation of China (Grant numbers 10774150 and 10834014) 
and the China 973-Program (Grant number 2007CB935903). 



References 

[1] Kirkpatrick S and Selman B 1994 Science 264 1297-1301 

[2] Achlioptas D 2001 Theor. Corn-put. Sci. 265 159-185 

[3] Monasson R and Zecchina R 1996 Phys. Rev. Lett. 76 3881-3885 

[4] Mezard M, Parisi G and Zecchina R 2002 Science 297 812-815 

[5] Krzakala F, Montanari A, Ricci-Tersenghi F, Semerjian G and Zdeborova L 2007 Proc. Natl. Acad. Sci. USA 
104 10318-10323 

[6] Hartmann A K and Weigt W 2005 Phase Transitions in Combinatorial Optimization Problems (Weinheim, 
Germany: Wiley- VCH) 

[7] Mezard M and Montanari A 2009 Information, Physics, and Computation (New York, USA: Oxford Univ. 
Press) 

[8] Mertens S, Mezard M and Zecchina R 2006 Rand. Struct. Algorithms 28 340-373 

[9] Montanari A, Ricci-Tersenghi F and Semerjian G 2008 J. Stat. Mech.: Theor. Exp. P04004 

[10] Zhou H 2008 Phys. Rev. E 77 066102 

[11] Mezard M and Parisi G 2001 Eur. Phys. J. B 20 217-233 

[12] Mezard M and Montanari A 2006 J. Stat. Phys. 124 1317-1350 

[13] Semerjian G 2008 J. Stat. Phys. 130 251-293 

[14] Ardelius J and Zdeborova L 2008 Phys. Rev. E 78 040101(R) 

[15] Gardner E 1985 Nucl. Phys. B 257 [FS14] 747-765 

[16] Zhou H 2009 Criticality and heterogeneity in the solution space of random constraint satisfaction problems 

|arXiv:0911.4328| (Int. J. Mod. Phys. B, accepted) 
[17] Zhou H and Ma H 2009 Phys. Rev. E 80 066108 
[18] Zhou H 2010 Eur. Phys. J. B 73 617-624 
[19] Li K, Ma H and Zhou H 2009 Phys. Rev. E 79 031102 

[20] Wasserman S and Faust K 1994 Social Network Analysis: Methods and Applications (New York: Cambridge 
University Press) 

[21] Girvan M and Newman M E J 2002 Proc. Natl. Acad. Sci. USA 99 7821-7826 
[22] Zhou H 2003 Phys. Rev. E 67 041908 
[23] Zhou H 2003 Phys. Rev. E 67 061901 

[24] Jain A K and Dubes R C 1988 Algorithms for Clustering Data (Englewood Cliffs, NJ, USA: Prentice-Hall) 

[25] Barthel W and Hartmann A K 2004 Phys. Rev. E 70 066120 

[26] Zhou H 2005 New J. Phys. 7 123 

[27] Cavagna A 2009 Phys. Report 476 51-124 

[28] Ediger M D 2000 Annu. Rev. Phys. Chem. 51 99-128 

[29] Glotzer S C 2000 J. Non-Cryst. Solids 274 342-355 

[30] A random _R"-SAT formula can be represented by a bipartite graph (see, e.g., Ref. |31j). In this bipartite 
graph, a circular node represents a variable and a square node represents a constraint, an edge between a 
variable node i and a constraint node a means that variable i is involved in constraint a. In this bipartite 
graph, the distance d(i,j) between two variable nodes i and j is defined as the number of constraint nodes 



on a shortest-length path between i and j. For example, if i and j both are constrained by the same 
constraint, then d(i,j) = 1; if i and j are not constrained by the same constraint but i and a variable k 
are involved in constraint a and j and k are involved in constraint b, then d(i,j) — 2. 

[31] Kschischang F R, Frey B J and Loeliger H A 2001 IEEE Trans. Infor. Theor. 47 498-519 

[32] Franz S and Parisi G 1995 J. de Physique I 5 1401-1415 

[33] Franz S and Parisi G 1997 Phys. Rev. Lett. 79 2486-2489 

[34] Zdeborova L and Krzakala F 2010 Phys. Rev. B 81 224202 

[35] Obuchi T and Kabashima Y 2009 J. Stat. Mech.: Theor. Exp. P12014 



